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Abstract. Let G c GL(V) be a complex reductive group. Let G denote {(p e GL(V) \ pop = 
p for all p G C[F] G }. We show that, in general, G = G. In case G is the adjoint group of a 
f**«« ■ simple Lie algebra g, we show that G is an order 2 extension of G. We also calculate G for all 

representations of SL2. 
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|7| ; 1. Introduction 

Our base field is C, the field of complex numbers. Let G C GL(V) be a reductive group. Let 
G' = {tp G GL(V) I p o tp = p for all p G C[V] G }. Several authors have studied the problem 
of determining G'. If G is finite, then one easily sees that G' = G. Solomon [Sol05, S0IO6] 
has classified many triples consisting of reductive groups H C G and a G-module V such that 
C(V) = C(V) G (rational invariant functions). If G and H are semisimple, then this is the same 
thing as finding triples where we have equality of the polynomial invariants: C[V] = C[V] G . 
We show that for "general" faithful G-modules V we have that G = G'. We also compute G' 
for all representations of SL2. 

First we study the case that G is the adjoint group of a simple Lie algebra g. Our interest 
in this case is due to the paper of Rais [Rai07] where the question of determining G' is raised. 
The case that g = si n was also settled by him [Rai72], where it is shown that G'/G is generated 
by the mapping si n 3 X 1— > X 1 where X 1 denotes the transpose of X. In §2 we show that, 
^ ■ in general, G'/G is generated by the element — ip where i/j: g — > g is a certain automorphism 
^ ! of g of order 2. In the case of sl n , ip(X) = —X 1 , so that our result reproduces that of Rais. 

The computation of G' for g semisimple follows easily from the case that g is simple. In §3 
we prove our result that G = G' for general G and general G-modules V. In §4 we consider 
representations of SL 2 . 

We thank M. Rais for bringing his paper [Rai07] to our attention and we thank him and 
^ ' D. Wehlau for help and advice. 
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2. The adjoint case 

Proposition 2.1. Let g be a semisimple Lie algebra, so we have g = gi © • • • © g r where the 
gi are simple ideals. Let ip G G' . Then <p(Qi) = Qi for all i, and ip\ g . = ±o"j where Oi is an 
automorphism of g^. 

Proof. By a theorem of Dixmier [Dix79] we know that the Lie algebra of G' is ad(g) C gl(g)- 
Thus (f acts on ad(g) ~ g via an automorphism a where ^oadXo tp~ l = ada(X) for X G g. 
Since <p induces the identity on C[g] G , so does a, and it follows that o = Yli^i where o~i G 
Aut(gj), % = 1, . . . , r. By Schur's lemma, tpoa^ 1 restricted to Qi is multiplication by some scalar 
Aj G C*, i = 1, . . . ,r. Since Aut(gj) and G' preserve the invariant of degree 2 corresponding to 
the Killing form on each g, we must have that \ = ±1, % = 1, . . . , r. □ 
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From now on we assume that g is simple. Let a £ Aut(g). Then we know that, up to 
multiplication by an element of G = Aut(g) , we can arrange that a preserves a fixed Cartan 
subalgebra t C g. Thus we may assume that <p preserves t. Let T denote the corresponding 
maximal torus of G. 

Corollary 2.2. We may modify ip by an element of G so that ip is the identity on t. 

Proof. By Chevalley's theorem, restriction to t gives an isomorphism of C[g] G with C[t] w/ where 
W is the Weyl group of g. Thus the restriction of to t coincides with an element of W, where 
every element of W is the restriction of an element of G stabilizing t. Thus we may assume 
that (p is the identity on t. □ 

Let $ be the set of roots and $ + a choice of positive roots. Let II denote the set of simple 
roots. Since (p = ±cr is the identity on t, o~(x) = c a x for all x £ t where c a — ±1. Hence either 
a sends each g a to itself or it sends each Q a to g_ Q , a £ $. Choose nonzero elements x a £ g a , 
a £ II, and choose elements y a £ g_ a such that (x a ,y a , [x a ,y a ]) is an s[ 2 -triple. Let ip denote 
the unique order 2 automorphism of g such that i/)(x) = —x, x £ t and ip(x a ) = —y a , a £ II 
(see [Hum72, 14.3]). 

Proposition 2.3. (1) If c a = 1, then o is inner. 

(2) If c a = —1, then a differs from ip by an element of Ad(T). 

Proof. If c a = 1, then a(x a ) = c a x a , c a £ C, a £ IT. There is a t £ T such that Ad(t)(x a ) = 
c a x a , a £ II. It follows that o = Ad(t) £ G. If c a = — 1, we can modify a by an element of T 
so that it becomes ip. □ 

Proposition 2.4. Let g be simple. Then the following are equivalent. 

(1) Every representation of g is self-dual. 

(2) The automorphism ip is inner. 

(3) The generators of C[g] G /icwe even degree. 

(4) g is q/ £/ie following type: 

(a) B n , n > 1, 

(b) C n , n > 3, 

(c) D 2n , n > 2, 

(d) E 7 , E 8 , F 4 or G 2 . 

Proof. The equivalence of (1), (3) and(4) is well-known. Now given a highest weight vector A 
of g, the highest weight vector of the corresponding dual representation V(A)* is — p(A) where 
p is the unique element of the Weyl group W which sends $ + to $~ ([Hum 72, §21, Exercise 
6]. Suppose that we have (2). Then, since i/j is inner and it normalizes t, it gives an element of 
W, namely p, so that V(A)* ~ ^(A) for all A and (1) holds. Conversely, if (1) holds, then — p 
is the identity on the set of weights, hence p(a) = —a for all a £ <&. It follows that p o if; is an 
automorphism of g which is the identity on t and sends g Q to g a for all a. Then p o %p £ Ad(T) 
so that ip is inner. □ 

Theorem 2.5. The group G'/G has order 2, generated by —ip. 

Proof. If ip = a £ Aut(g), then Proposition 2.3 shows that <p> = a £ G. If <p = —a, then by 
Proposition 2.3 we may assume that <p = — Now —ip induces an automorphism of C[g] G and 
— ip is the identity on t. Hence Chevalley's theorem shows that — ip £ G' and we know that —ip 
generates G'/G. Moreover, —ip is not in Aut(g), so that —ip £" G. □ 

Corollary 2.6. Suppose that ip is inner. Then G'/G is generated by multiplication by —1. 

We leave it to the reader to formulate versions of Theorem 2.5 and Corollary 2.6 for the 
semisimple case. 
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3. The general case 

We have a finite dimensional vector space V^and G is a reductive subgroup of GL(V). Let 
G' := {ip G GL(V) | po if = p for all p G C[V] G }. We show that, "in general," we have G' = G. 

Let U denote the subset of V consisting of closed G-orbits with trivial stabilizer. It follows 
from Luna's slice theorem [Lun73] that U is open in V. 

Theorem 3.1. Suppose that V \ U is of codimension 2 in V . Then G' — G. 

Proof. Let tp G G' and let x G U. Then <p(x) = ip(x) ■ x where ip: U — > G is a well-defined 
morphism. Since G is affine, we may consider ip as a mapping from U -^GcC" for some n 
where G is Zariski closed in C n . Our condition on the codimension of V \ U guarantees that 
each component of ip is a regular function on V, hence ip extends to a morphism defined on all 
of V, with image in G. Now let x G U. Then 

tp(x) = \imip(tx)/t = lim ip(tx)tx/t = ip(0)(x). 
Thus ip is just the action of ip(0) G G, so G' — G. □ 

4. Representations of SL 2 

As an illustration, we consider representations of G — SL 2 or G = S0 3 . We only consider 
representations with no nonzero fixed subspace. We let Rj denote the irreducible representation 
of dimension j+1, j > 0, and kRj denotes the direct sum of k copies of k > 1. When we have 
a representation only containing copies of Rj for j even, then we are considering representations 
of SO3. From [Sch95, 11.9] we know that all representations of G satisfy the hypotheses of 
Theorem 3.1 except for the following cases, where we compute G'. 

(1) For Ri we have G' = GL 2 , for 2Rx we have G 1 = O4 and for 3Ri we have G' = G. 

(2) For R 2 we have G' = 3 and for 2R 2 we have G' = 3 . (Here G = S0 3 .) 

(3) For R 2 © R x we have G' = {cf G GL 2 | det(p') = ±1}. 

(4) For i? 3 the group G' is the same as in case (3). 

(5) For R A we have G' = G = S0 3 . 

Most of the calculations are easy, we mention some details for some of the non obvious cases. 

Suppose that our representation is -R4, which has generating invariants of degrees 2 and 3. 
The Lie algebra q' acts irreducibly on R A , hence it is the sum of a center and a semisimple 
Lie algebra [Jac62, Ch. II, Theorem 11]. Clearly we cannot have a nontrivial center, so that 
g' is semisimple. Now a case by case check of the possibilities forces $j' = g. Suppose that 
g' G G'\ G. Then conjugation by g' gives an inner automorphism of G, hence we can correct g' 
by an element of G so that g' commutes with G. Thus g' acts on i? 4 as a scalar. But to preserve 
the invariants the scalar must be 1. Thus we have G' = G. Similar considerations give that 
q' = g in case (4), so that G'/G is generated by scalar multiplication by % (since the generating 
invariant of R3 has degree 4), which shows that G' is as claimed. 

In case (3), one sees that g' = g, so that generators of G'/G act as scalars on R 2 and R\. 
Now generators of the invariants have degrees (2, 0) and (1, 2) so that G'/G is generated by an 
element which is multiplication by —1 on R 2 and multiplication by i on R±. Hence G' is as 
claimed. 
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